Size Change: 0 B

Total Size: 1.4 MB

_{compressed-size-action}

I managed to get e2e tests working in rootless docker with this change on my machine, but I'm getting this error on GitHub doing the same thing.

 Step 10/25 : RUN mkdir -p /var/www/html/wp-content/uploads/
 ---> Running in b8ef0832edac
Removing intermediate container b8ef0832edac
 ---> 2c04e9c8799a
Step 11/25 : RUN chown -R $HOST_UID:$HOST_GID /var/www/html/wp-content/uploads/
 ---> Running in 30b7b5997c44
chown: cannot access '/var/www/html/wp-content/uploads/': No such file or directory

@noahtallen I'm no Docker expert, is there any reason this would be failing here? Or is there a better way to change permissions in Docker?

Hm, I'm no expert either, but throwing out some ideas:

• Is chown actually required after running mkdir? (Wouldn't the user be the same?)
• Why would we need to create the directory if it included in the docker-compose mount? (see the home mount for an example?)

I'm also curious, what errors do you see in rootless mode?

I also don't have a good way to test rootless mode unfortunately. thanks for opening a PR!

Is chown actually required after running mkdir? (Wouldn't the user be the same?)

The DOCKERFILE is running the commands as root, so chown is necessary to move ownership to the host user.

Why would we need to create the directory if it included in the docker-compose mount?

That's what I started with, but I kept getting the same error until I created the dir and chowned it.

See the home mount for an example?

The -m flag on useradd creates the user's home directory with the right permissions.

I'm also curious, what errors do you see in rootless mode?

Updated the description. I should have had that added before :)

Flaky tests detected in c44b630.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/5049246189
📝 Reported issues:

• [Flaky Test] can be replaced by dragging-and-dropping images from the inserter #50325 in /test/e2e/specs/editor/blocks/image.spec.js
• [Flaky Test] should apply Heading block styles to all Heading blocks #49942 in /test/e2e/specs/site-editor/push-to-global-styles.spec.js
• [Flaky Test] should delete the template after saving the reverted template #50783 in /test/e2e/specs/site-editor/template-revert.spec.js
• [Flaky Test] should disappear when closed via click event or Escape key #50807 in /test/e2e/specs/site-editor/style-book.spec.js
• [Flaky Test] should show the original content after revert #47897 in /test/e2e/specs/site-editor/template-revert.spec.js

I made it past wp-env start by using install rather than mkdir and chown. TIL you can create directories with the -d flag of install, and for some reason that works better here. 🤷

The tests are all consistently passing locally, but consistently failing on GitHub. I'm not really sure how to debug this.

Also, #32519 should actually already be fixed. The uploads are now coming from the locally mapped WordPress installation, so, they persist unless you explicitly run wp-env destroy.

Thanks for the quick review @ObliviousHarmony!

What if we set the userns_mode to host in the docker-compose.yml file?

I just tried that on a fresh branch off of trunk, and I still end up with the original error where uploads aren't allowed.

Given the existence of this particular problem, however, I think there will be other permission problems when running Docker in rootless mode to consider.

I was wondering if we should set up permissions as described in the Hardening WordPress docs, but I had something working locally with just the uploads change and figured it would be better to start small and fix my one issue first.

I just tried that on a fresh branch off of trunk, and I still end up with the original error where uploads aren't allowed.

We did some further debugging together and weren't able to find a way to make this work. I did some further research and it looks like this is unfortunately unavoidable. If you're running in rootless then using your host uid and gid will give permission errors.

I was wondering if we should set up permissions as described in the Hardening WordPress docs, but I had something working locally with just the uploads change and figured it would be better to start small and fix my one issue first.

Honestly, I would prefer if we avoided added any permission hacks specific for Rootless Docker. There's a problem here in how permissions work between these two deployments, but, I've got an idea!

In #50814 (comment) it was highlighted that there's a problem with Windows caused by the usage of root's UID. In fixing this we can support running Docker internally as uid 0, so, what if we did that here? Since we have getHostUser(), what if we used something like docker context inspect to detect headless mode and then used a uid and gid of 0 in that case? It would map to the host user in the same way and so there would be no permission issues.

Let me take a pass at solving both of these problems in one go. Could you share the output of docker context inspect @ajlende so I can take a look at that?

Could you share the output of docker context inspect @ajlende so I can take a look at that?

[
    {
        "Name": "default",
        "Metadata": {},
        "Endpoints": {
            "docker": {
                "Host": "unix:///run/user/1000/docker.sock",
                "SkipTLSVerify": false
            }
        },
        "TLSMaterial": {},
        "Storage": {
            "MetadataPath": "\u003cIN MEMORY\u003e",
            "TLSPath": "\u003cIN MEMORY\u003e"
        }
    }
]

What about docker info @ajlende?

In #50814 (comment) it was highlighted that there's a problem with Windows caused by the usage of root's UID. In fixing this we can support running Docker internally as uid 0, so, what if we did that here? Since we have getHostUser(), what if we used something like docker context inspect to detect headless mode and then used a uid and gid of 0 in that case? It would map to the host user in the same way and so there would be no permission issues.

Nice, I love the idea of solving both problems at once!

What about docker info @ajlende?

$ docker info

Client:
 Version:    24.0.0
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  2.18.1
    Path:     /usr/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 1
 Server Version: 24.0.0
 Storage Driver: fuse-overlayfs
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1677a17964311325ed1c31e2c0a3589ce6d5c30d.m
 runc version: 
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  rootless
  cgroupns
 Kernel Version: 6.3.2-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 31.27GiB
 Name: astrid
 ID: LQHD:OW2Z:L4TD:EDN4:KR6N:7VTY:45MA:GI2Z:ICE5:VHT3:HWJJ:PNYJ
 Docker Root Dir: /home/ajlende/.local/share/docker
 Debug Mode: true
  File Descriptors: 24
  Goroutines: 39
  System Time: 2023-05-23T12:42:59.600690909-05:00
  EventsListeners: 0
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

It does seem to list rootless under Security Options.

I ran into what should have been a pretty obvious problem with running everything as root 😅 Everything gets really mad and the lengths we have to go to in order to make that work are pretty extreme.

I'll let @noahtallen weigh in, but, I don't think rootless Docker is something we should try and go to extreme lengths to support. Ultimately it boils down to everything being owned by root again and the various problems that can cause. Although we can fix the particular issue with uploads, we're still going to see problems elsewhere in cases where the filesystem gets manipulated in any way. A lot of folks I know who used Linux previously avoided wp-env like the plague because of compatibility problems and I think the same problems exist here when using rootless Docker.

Ideally userns_mode would have let us disable user mapping, but, that doesn't seem to be the case. These kinds of problems seem relatively common with using rootless Docker and mapped directories unfortunately. I think the best course of action here would be to use wp-env run with sudo commands to make any permission-related changes you need to get things working for you locally.

I mostly agree with @ObliviousHarmony. I don't think wp-env should try super hard to support this case, but at the same time, if there is a simple fix to certain issues, I think we should include it. But if rootless mode is generally incompatible with the way wp-env & docker compose work with mounts and WordPress, it sounds like it'd be difficult for us to support properly.

Thanks for the help and teaching me some things about wp-env. Since this isn't as simple as I first thought, we'll close this PR, and I'll take @ObliviousHarmony's suggestion to just modify my containers manually for now.